-
-
Notifications
You must be signed in to change notification settings - Fork 33.9k
gh-143637: Fix re-entrant mutation of ancillary data in socket.sendmsg() #143892
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
gh-143637: Fix re-entrant mutation of ancillary data in socket.sendmsg() #143892
Conversation
|
Hi, this PR fixes an internal correctness issue in socket.sendmsg(). |
|
No. It remains uaerfacing. In general, every bug fix should be announced if it can be seen only with pure python code and the public API |
Modules/socketmodule.c
Outdated
| #else | ||
| if (!get_CMSG_LEN(bufsize, &space)) { | ||
| #endif | ||
| if(!get_CMSG_SPACE(bufsize, &space)){ |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why this change??
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This value is only used to decide how much memory to allocate for msg.msg_control.
get_CMSG_LEN() can give a smaller size on some systems because it doesnt include extra padding that the OS needs which means the buffer can be too small.
get_CMSG_SPACE() includes this padding, so it correctly calculates how much space is needed. This function already uses get_CMSG_SPACE() elsewhere for the same reason, so using it here keeps the logic consistent and safe.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The function is not necessarily present though?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
get_CMSG_SPACE() here is not the platform macro. It’s a CPython helper defined in socketmodule.c itself, so it is always available when this code is compiled. Internally, that helper already handles platform differences and falls back appropriately when CMSG_SPACE is not provided by the system. So using get_CMSG_SPACE() unconditionally here doesn’t introduce a new dependency, it just reuses the same helper that this file already relies on in other places.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I still do not understand why this was modified in this PR? is it affecting the issue? has it been the LLM who decided on it? never use LLM for writing your code if you do not understand why it wrote this or that.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
after revisiting the issue, I realizd it doesn’t actually relate to the re-entrant mutation bug. I’ve reverted it and kept the PR focused. Thanks, this helped me understand the problem much better.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can you answer questions when I ask them? I asked whether it was an LLM who did this. If so, stop using an LLM without understanding its changes as it is against our rules. I will also close PRs that are LLM-aided but not understood.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I should have answered that directly I am sorry about that, yes I used LLM, without fully understanding that was a mistake on my part.
I understand your concern and won’t include changes I can’t clearly justify going forward.
Modules/socketmodule.c
Outdated
| goto finally; | ||
| } | ||
| controllen += space; | ||
| controllen+=space; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why this change?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Good catch thanks, this line change is incidental and not required for the fix.
I’ll revert it.
Lib/test/test_socket.py
Outdated
| # involve recvmsg() or recvmsg_into(). | ||
| @unittest.skipUnless(hasattr(socket.socket, "sendmsg"), | ||
| "sendmsg not supported") | ||
| def _test_sendmsg_reentrant_ancillary_mutation(self): |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why a private method?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is a helper method rather than a standalone test because it depends on the surrounding test class setup and socket lifecycle, and is not meant to be picked up directly by unittest discovery.
The skipUnless(sendmsg) decorator is placed here so the helper only runs on platforms that support sendmsg(),with execution still controlled by the enclosing test logic.
I can add a small public test_* wrapper calling this helper if that’s prefferable.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
But where is it tested then?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sorry for the confusion, you are right, I have missed it- I have added a public test method that invokes _test_sendmsg_reentrant_ancillary_mutation in my updated PR is it correct?, Thanks for the clarification.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Just rename _test_sendmsg_reentrant_ancillary_mutation() to test_sendmsg_reentrant_ancillary_mutation().
|
A Python core developer has requested some changes be made to your pull request before we can consider merging it. If you could please address their requests along with any other requests in other reviews from core developers that would be appreciated. Once you have made the requested changes, please leave a comment on this pull request containing the phrase |
|
Thanks for guiding, I have added the news entry. |
|
I have made the requested changes; please review again. |
|
Thanks for making the requested changes! @picnixz: please review the changes made to this pull request. |
…nshu2282-cyber/cpython into fix-sendmsg-reentrant-cmsg
Lib/test/test_socket.py
Outdated
| # involve recvmsg() or recvmsg_into(). | ||
| @unittest.skipUnless(hasattr(socket.socket, "sendmsg"), | ||
| "sendmsg not supported") | ||
| def _test_sendmsg_reentrant_ancillary_mutation(self): |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Just rename _test_sendmsg_reentrant_ancillary_mutation() to test_sendmsg_reentrant_ancillary_mutation().
Modules/socketmodule.c
Outdated
| } | ||
| ncmsgs = PyTuple_GET_SIZE(cmsg_fast); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The indentation looks wrong.
Modules/socketmodule.c
Outdated
| "(iiy*):[sendmsg() ancillary data items]", | ||
| &cmsgs[ncmsgbufs].level, | ||
| &cmsgs[ncmsgbufs].type, | ||
| &cmsgs[ncmsgbufs].data)){ |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The indentation looks wrong, it was correct in the existing code.
Lib/test/test_socket.py
Outdated
| self._test_sendmsg_reentrant_ancillary_mutation() | ||
|
|
||
| def _test_sendmsg_reentrant_ancillary_mutation(self): | ||
| import socket |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
it's already imported at the module top level.
Lib/test/test_socket.py
Outdated
| def _test_sendmsg_reentrant_ancillary_mutation(self): | ||
| import socket | ||
|
|
||
| seq = [] |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
no need to declare it before Mut, you can declare after Mut declaration.
Lib/test/test_socket.py
Outdated
|
|
||
| class Mut: | ||
| def __init__(self): | ||
| self.tripped = False |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't think that we need this tripped flag machinery, just always call seq.clear() in __index__(), no?
Lib/test/test_socket.py
Outdated
| pass | ||
|
|
||
|
|
||
| class SendmsgReentrancyTests(unittest.TestCase): |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why not adding the test to SendmsgStreamTests?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I have tried it but the test are failing so I decided it move it into a seperate class.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What was the error?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
AttributeError: 'SendmsgUnixStreamTest' object has no attribute '_test_sendmsg_reentrant_ancillary_mutation'. Did you mean: 'test_sendmsg_reentrant_ancillary_mutation'?
AttributeError: 'SendmsgTCPTest' object has no attribute '_test_sendmsg_reentrant_ancillary_mutation'. Did you mean: 'test_sendmsg_reentrant_ancillary_mutation'?
got these two errors instead removing private method completely.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
According to what I got is SendmsgStreamTests the class adds _ prefix internally before the method name, just to confirm. I mean by looking the class, Is this class expects _test_* implementation for each test_* method.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ah yes, these tests use one function for the server, and one function for the client. Since you expect an error before actually sending data to the peer, I suggest moving this test to the generic GeneralModuleTests class.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for the clarification, I have made the test available in GeneralModuleTests class. Happy to make any further adjustments as needed.
Co-authored-by: Victor Stinner <[email protected]>
Co-authored-by: Victor Stinner <[email protected]>
Fix a crash in socket.sendmsg() caused by re-entrant mutation of ancillary data during argument parsing.
Hold a strong reference to each ancillary item while parsing to avoid use-after-free, and added a regression test.
socket.sendmsgancillary parser after re-entrant__index__clears the control list #143637